Using Amazon S3 and Blogger for family history

The easiest way to share large scans of family history journals is through the internet, using a combination of Amazon S3 (for storage) and Blogger (to display and present the files). Getting the system set up is relatively easy; once it’s set up it’s even easier to use.

Amazon S3

Besides selling everything in the world, Amazon offers a ton of web services: cloud computing, database management, content delivery networks, and other nerdy development things. One of their best and most convenient offerings is S3, or Simple Storage Service—essentially a giant hard drive in the cloud. Storage costs are minimal ($0.095 per GB), so you can store a ton of stuff safely online.

To get started, go to http://aws.amazon.com/s3/ and create a free account.

Create a new account

Log in with your Amazon account and follow the instructions. Eventually you’ll come to your Amazon Web Services console at https://console.aws.amazon.com/console/home, where you can access the dashboard for S3

S3 on the AWS console

S3 lets you create an unlimited number of “buckets” for storage. Buckets aren’t quite like folders—they’re more like actual hard drives. As you can see in the screenshot below, I have buckets for a bunch of different things, like blog files (where we upload our Christmas newsletter), and other stuff. You can create as many buckets as you want at no cost—you’re only charged for storage space. So for this you’d probably want just one bucket for all the family history stuff you’re doing, with multiple folders inside that bucket. Click on “Create Bucket” to create a bucket (obviously).

Create a new bucket

Give your bucket a unique name (‘familyhistory’ didn’t work for me) and click “Create.” You don’t need to worry about the “Set Up Logging” thing.

New bucket dialog

You should now have a new bucket in your bucket list (pseudo pun!).

New bucket in the list

Click on the new bucket to see its contents. Nothing’s there! Click on “Create Folder” to, um, create a folder.

Create a new folder

By default, the folders that you create will be private and only accessible by you (since lots of companies use S3 to store user data). You’ll need to make any new folders publicly accessible by right clicking on them and choosing “Make Public”:

Make that folder public

Now anything you put in that folder will have a publicly accessible URL. Magic.

That’s it! You’ve set up S3! Now you need to put stuff in those folders.

Add files to S3

Now that you’ve set up S3, you have everything you need to upload files and share their URLs. There are two general ways to add files to your S3 bucket: (1) using the S3 website and (2) using a 3rd-party program. Both are really easy to do, and both let you quickly get publicly accessible and shareable URLs.

Use the S3 website

Log into the your S3 console at https://console.aws.amazon.com/s3/. Open the bucket and folder you want to upload to. Then click “Upload”:

Upload file to S3

Click “Add Files” to select the files on your computer that you want to upload, then click “Start Upload.” Alternatively, you can click the “Enable Enhanced Uploader (BETA)” link to upload whole folders. I’ve never done this, but it probably works.

Upload dialog

You can then look in the folder to see a list of the files you’ve uploaded. If you click on one of the files, you’ll get a screen of summary information, including a publicly accessible URL.

File properties

Copy that, paste it in a browser tab, and the file will download. Send that link to anyone and they can download it. Paste that link into a blog post and people can click it and download the file. Magic.

Use a 3rd-party program

Rather than use the S3 website, I find it easier to use a different program that lets you drag and drop files from your computer to the S3 server and quickly get their URLs. The best (and free-est) program is called Cyberduck. Go to http://cyberduck.ch/ and download the Windows version and install it on your computer.

Download Cyberduck

Once it’s installed, you can use Cyberduck to connect to S3. The best way to do this is to save your connection information inside Cyberduck as a bookmark. Click on Bookmark > New Bookmark to create one.

Create new bookmark

In the new bookmark dialog box, change the “FTP (File Transfer Protocol)” to “S3 (Amazon Simple Storage Service)”

Select S3 from the list

Give the bookmark whatever nickname you want. Leave the server information as is. You can find your access key ID at the S3 website. Click on your name in the top right corner and select “Security Credentials” from the menu.

Security credentials

If you get some warning or message about AWS Identity and Access Management, ignore it and click “Continue to Security Credentials”.

Click on Access Keys and click on “Create New Root Key” to generate a new access key ID and secret access key. Copy these and save them somewhere safe.

New credentials

Paste the access key ID into the Cyberduck bookmark and then close the bookmark editing dialog (for some reason it won’t let you enter the password until you try to connect to the server).

Finish the bookmark

Once the bookmark dialog is closed, you’ll see the bookmark in the list of bookmarks. Double click on it to connect. Since this is your first time connecting, it’ll ask for your password. Paste in the huge crazy long secret access key, make sure “Save password” is checked, and click “Login”

Log in

You’ll see a list of your buckets, just like on the website. Double click on the bucket you created to open it and see the folders.

To upload stuff, just drag files from your computer into folders in Cyberduck. You can even drag entire folders.

Getting a URL for a file in Cyberduck is super easy. Right click on a file, go to “Copy URL” and choose “HTTP URL” (all the others are for other things you don’t need)

Copy the file’s URL

Sometimes when you reopen Cyberduck, it’ll try to open the last folder it was in. If you want to get back to the list of bookmarks, click on the little bookmark icon near the top:

Bookmark icon

There’s only one caveat! Folders in S3 are private by default. If you drag a folder from your computer into the main root of the bucket (i.e. not into a folder you’ve already set as public), it’ll be private. You can either (1) log into the S3 website and right click on the folder to make it public, or (2) upload stuff to an already public folder. An easy way around this would be to create one main folder for uploaded stuff within your bucket and upload all other files and folders there—then anything you upload will automatically be public as well.

Putting it all together

Now that you’ve done the hard steps of setting this up, here’s how you can use everything in the future.

Using Cyberduck

  1. Open Cyberduck.
  2. Double click on the S3 connection in the bookmark list.
  3. Navigate to the folder where you want to upload stuff.
  4. Drag files from your computer into Cyberduck.
  5. Right click on the files to get their public URLs.
  6. Do something with those URLs.

Using the S3 website

  1. Log into S3 at https://console.aws.amazon.com/s3/
  2. Navigate to the folder where you want to upload stuff.
  3. Upload files.
  4. Click on the files and find their URLs in the file properties screen.
  5. Do something with those URLs.

Distribute the URLs

Once your files are in the cloud on S3, all you need to do is get their URLs and distribute them somehow.

Ideally you could then make some curated blog posts in Blogger, like this:

Joseph Stacey Murdock was born in X in Y. He had Z kids. Blah blah. He left several personal journals. The first (http://the-url-for-the-first-journal) was found in 1893 and talks about stuff. The second (http://the-url-for-the-second-journal) was found later and talks about other stuff. These are cool.

Alternatively, if someone in your family wants a specific journal or scan, you can open Cyberduck, find the file, copy the URL and send it to them (rather than tracking down the specific blog post).

That’s all!